Information processing system, program, and information processing method

ABSTRACT

An information processing system includes one or more vehicles, and a server communicable with the one or more vehicles. The vehicle acquires an image obtained by imaging a road on which a host vehicle is located. The vehicle or the server determines a degree of difficulty in traveling on the road due to snow cover from the image. The server stores the degree of difficulty in traveling for each of one or more roads, and provides information to a client by using the stored degree of difficulty in traveling for each of one or more roads.

INCORPORATION BY REFERENCE

This application is a continuation of U.S. application Ser. No.16/701,432, filed Dec. 3, 2019, which claims the benefit of priorityfrom Japanese Patent Application No. 2018-238715, filed Dec. 20, 2018,the entire contents of which are incorporated herein by reference.

BACKGROUND 1. Technical Field

The disclosure relates to an information processing system, a program,and an information processing method.

2. Description of Related Art

In the related art, a technology for acquiring information on snow coverhas been known. For example, Japanese Unexamined Patent ApplicationPublication No. 2013-061726 (JP 2013-061726 A) discloses a technology inwhich an in-vehicle camera mounted on a vehicle images snowfall tovisually observe the snowball amount and acquires snowfall amountmeasurement data by analyzing the imaged image data.

SUMMARY

However, the technology disclosed in JP 2013-061726 A is based on thepremise that snowfall is captured in the image of the in-vehicle camera.In this way, for example, snowfall amount measurement data cannot beacquired in a place where there is no snowfall. Therefore, there is roomfor improvement in the related art for acquiring information on snowcover on a road.

An object of the disclosure made in consideration of the abovecircumstances is to improve a technology for acquiring information onsnow cover on a road.

A first aspect of the disclosure relates to an information processingsystem including one or more vehicles and a server communicable with theone or more vehicles. The vehicle is configured to acquire an imageobtained by imaging a road on which a host vehicle is located. Thevehicle or the server is configured to execute determination processingfor determining a degree of difficulty in traveling on the road due tosnow cover from the image. The server is configured to store the degreeof difficulty in traveling for each of one or more roads and provideinformation to a client by using the stored degree of difficulty intraveling for each of the one or more roads.

A second aspect of the disclosure relates to a program. The programcauses a vehicle communicable with a server to execute steps ofacquiring an image obtained by imaging a road on which a host vehicle islocated and determining a degree of difficulty in traveling on the roaddue to snow cover from the image.

A third aspect of the disclosure relates to an information processingmethod executed by an information processing system including one ormore vehicles and a server communicable with the one or more vehicles.The method includes acquiring, by the vehicle, an image obtained byimaging a road on which a host vehicle is located, determining, by thevehicle or the server, a degree of difficulty in traveling on the roaddue to snow cover from the image, storing, by the server, the degree ofdifficulty in traveling for each of one or more roads, and providing, bythe server, information to a client by using the stored degree ofdifficulty in traveling for each of the one or more roads.

With the information processing system, the program, and the informationprocessing method according to the aspects of the disclosure, thetechnology for acquiring information on snow cover on road is improved.

BRIEF DESCRIPTION OF THE DRAWINGS

Features, advantages, and technical and industrial significance ofexemplary embodiments of the disclosure will be described below withreference to the accompanying drawings, in which like numerals denotelike elements, and wherein:

FIG. 1 is a diagram showing a schematic configuration of an informationprocessing system according to an embodiment of the disclosure;

FIG. 2 is a block diagram showing a schematic configuration of avehicle;

FIG. 3 is a diagram showing an image obtained by imaging a sceneryoutside a vehicle;

FIG. 4 is a block diagram showing a schematic configuration of a server;

FIG. 5 is a diagram showing an example of information stored in theserver;

FIG. 6 is a diagram showing an example of providing information from theserver to a client;

FIG. 7 is a flowchart showing an operation of the vehicle; and

FIG. 8 is a flowchart showing an operation of the server.

DETAILED DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present disclosure will be described.

Configuration of Information Processing System

An overview of an information processing system 1 according to anembodiment of the present disclosure will be described with reference toFIG. 1. The information processing system 1 includes one or morevehicles 10 and a server 20. The vehicles 10 are, for example,automobiles, but they are not limited thereto and may be arbitraryvehicles. In FIG. 1, for convenience of description, only two vehicles10 are illustrated, but the information processing system 1 may includeany number of vehicles 10. The server 20 includes one or a plurality ofinformation processing devices (for example, server devices) that cancommunicate with each other. The vehicle 10 and the server 20 cancommunicate through a network 30 including, for example, a mobilecommunication network and the Internet. Further, the server 20 cancommunicate with a client 31 through the network 30. The client 31 is,for example, a personal computer (PC), a server device, a smartphone, ora vehicle, but may be any information processing device. The vehicle 10according to the embodiment may be the client 31.

The outline of the embodiment will be described first, and the detailsthereof will be described later. The vehicle 10 acquires an imageobtained by imaging a road on which a host vehicle is located, forexample, by using an in-vehicle camera. Here, when the vehicle 10 islocated in a snow cover area or a snowfall area, snow cover may bepresent on the road on the image. In general, the degree of difficultyin traveling on a road varies depending on snow cover conditions. Thevehicle 10 determines, from the image, the degree of difficulty intraveling on the road due to snow cover, and transmits it to the server20. The server 20 receives the degree of difficulty in traveling foreach of one or more roads from one or a plurality of vehicles 10 andstores the degree of difficulty in traveling. Then, the server 20provides information to the client 31 by using the stored degree ofdifficulty in traveling for each of the one or more roads.

As described above, according to the embodiment, the degree ofdifficulty in traveling on the road due to snow cover is determined byusing the image actually imaged by the vehicle 10. The degree ofdifficulty in traveling determined from the actual image is highlyaccurate information that is more in line with the actual situation inthe field than, for example, the degree of difficulty in travelingestimated from weather information. Therefore, the technique foracquiring information on snow cover on a road is improved.

Next, each configuration of the information processing system 1 will bedescribed in detail.

Configuration of Vehicle

As shown in FIG. 2, the vehicle 10 includes a communication unit 11, apositioning unit 12, an imaging unit 13, a storage unit 14, and acontroller 15. The communication unit 11, the positioning unit 12, theimaging unit 13, the storage unit 14, and the controller 15 may berespectively built in the vehicle 10 or may be respectively provided inthe vehicle 10 in a detachable manner. The communication unit 11, thepositioning unit 12, the imaging unit 13, the storage unit 14, and thecontroller 15 are connected to each other in a communicable mannerthrough, for example, an on-vehicle network such as controller areanetwork (CAN) or a dedicated line.

The communication unit 11 includes a communication module connected tothe network 30. The communication module is compatible with mobilecommunication standards such as 4th Generation (4G) and 5th Generation(5G), but is not limited thereto, and may be compatible with anycommunication standard. For example, an on-vehicle communicationapparatus such as data communication module (DCM) may function as thecommunication unit 11. In the embodiment, the vehicle 10 is connected tothe network 30 through the communication unit 11.

The positioning unit 12 includes a receiver compatible with a satellitepositioning system. The receiver is compatible with, for example, aglobal positioning system (GPS), but is not limited thereto, and may becompatible with any satellite positioning system. The positioning unit12 includes, for example, a gyro sensor and a geomagnetic sensor. Forexample, a car navigation device may function as the positioning unit12. In the embodiment, the vehicle 10 acquires the position of a hostvehicle and a direction in which the host vehicle is facing by using thepositioning unit 12.

The imaging unit 13 includes an in-vehicle camera that generates animage obtained by imaging a subject in the field of view. The image maybe a still image or a moving image. The in-vehicle camera may be amonocular camera or a stereo camera. The imaging unit 13 is provided inthe vehicle 10 such that the road on which the vehicle 10 is located canbe imaged. More specifically, the imaging unit 13 may be provided in thevehicle 10 such that a road in front of the vehicle 10 can be imaged.For example, an electronic apparatus having a camera function such as adrive recorder or a smartphone used by an occupant may function as theimaging unit 13. In the embodiment, the vehicle 10 acquires an imageobtained by imaging a road on which a host vehicle is located by usingthe imaging unit 13.

The storage unit 14 includes one or more memories. In the embodiment,the “memory” is, for example, a semiconductor memory, a magnetic memory,or an optical memory, but is not limited thereto. Each memory includedin the storage unit 14 may function as, for example, a main storagedevice, an auxiliary storage device, or a cache memory. The storage unit14 stores predetermined information used for the operation of thevehicle 10. For example, the storage unit 14 may store a system program,an application program, embedded software, road map information and thelike. The road map information includes, for example, roadidentification (ID) and junction ID allowing roads (links) and junctions(nodes) between roads to be uniquely identified, respectively, andposition information on the road and the junction. The informationstored in the storage unit 14 may be updatable with, for example,information to be acquired from the network 30 through the communicationunit 11.

The controller 15 includes one or more processors. In the embodiment,the “processor” is a general-purpose processor, a dedicated processorspecialized for specific processing, or the like, but is not limitedthereto. For example, an electronic control unit (ECU) mounted on thevehicle 10 may function as the controller 15. The controller 15 has atime measuring function for grasping the current time. The controller 15controls the operation of the entire vehicle 10.

For example, the controller 15 acquires weather information indicating asnow cover area or a snowfall area. Specifically, the controller 15 mayacquire weather information from any node on the network 30 storing theweather information, for example, an organization, such as theMeteorological Agency, or the server 20, through the communication unit11.

Moreover, the controller 15 determines whether or not the host vehicleis located in the snow cover area or the snowfall area indicated by theacquired weather information. When determination is made that the hostvehicle is located in the snow cover area or the snowfall area, thecontroller 15 acquires an image obtained by imaging the road on whichthe host vehicle is located. Here, when determination is made that thehost vehicle is in the snow cover area or the snowfall area, thecontroller 15 may acquire the image by imaging the scenery outside thevehicle by the imaging unit 13. Alternatively, the controller 15 maycause the imaging unit 13 to image a moving image of the road on whichthe host vehicle is located, and acquire an image by extracting theimage from the moving image when determination is made that the hostvehicle is in the snow cover area or the snowfall area. The controller15 acquires the time (imaging time) when the acquired image has beenimaged and the road ID of the road on which the host vehicle is located.The road ID can be acquired, for example, based on a comparison betweenthe location of the vehicle 10 (imaging position) at the imaging time ofthe image and the road map information stored in the storage unit 14.

Furthermore, the controller 15 executes determination processing fordetermining the degree of difficulty in traveling on the road due tosnow cover. In the embodiment, the higher the degree of difficulty intraveling, the more difficult the road traveling is. The degree ofdifficulty in traveling may be indicated by a grade (for example, “low”,“medium”, and “high”) or may be indicated by a numerical value (forexample, 1 to 100).

In determining the degree of difficulty in traveling, any method usingan image can be employed. In the embodiment, the controller 15 tries todetect a predetermined detection target from the image described above.The detection target is any target suggesting that the road is not in atraveling impossible state (for example, a state in which the road iscompletely covered with snow). In other words, the detection target isany target suggesting that the road is in a traveling possible state. Aspecific example of the detection target will be described below. Thecontroller 15 determines that the degree of difficulty in traveling onthe road is lower when at least one detection target is detected thanwhen no detection target is detected (for example, the road iscompletely covered with snow). In addition, the controller 15 maydetermine that the degree of difficulty in traveling is the highest whenno detection target is detected.

A specific example of the detection target will be described below. Forexample, as shown in FIG. 3, the above-mentioned image can include snowcover on the road (dots in the figure), an oncoming vehicle 41, a whiteline 42, tire traces 43, and snow 44 piled up beside the road. Whenanother vehicle traveling on the road (for example, the oncoming vehicle41 or a preceding vehicle) is shown in the image, since the othervehicle is actually traveling on the road, it can be said that the roadis highly likely to be in a travelable state. In addition, when a roadmarking on the road (for example, the white line 42 or a crosswalk) isshown in the image, since the road is not completely covered with snow,it can be said that the road is highly likely to be in the travelablestate. Furthermore, when the tire traces 43 are shown in the image,since it is considered that the preceding vehicle has traveled on theroad a short time ago, it can be said that the road is highly likely tobe in the travelable state. Furthermore, when snow 44 piled up besidethe road is shown in the image, since it is considered that snow removalwork has been carried out, it can be said that the road is highly likelyto be in the travelable state. For this reason, in the embodiment, whenat least one of the four detection targets, that is, another vehicletraveling on the road, the road marking on the road, tire traces on theroad, and the snow piled up beside the road is detected, the controller15 determines that the degree of difficulty in traveling is lower thanwhen none of the detection targets is detected. Any image recognitionalgorithm such as pattern matching, feature extraction, or machinelearning, can be employed for the detection of the detection targets.Furthermore, the detection target is not limited to the above example,and may be arbitrarily determined.

Further, the controller 15 transmits the above-described road ID, theimaging time, and the degree of difficulty in traveling to the server 20through the communication unit 11. Here, the controller 15 may furthertransmit the image to the server 20, for example.

Configuration of Server

As shown in FIG. 4, the server 20 includes a server communication unit21, a server storage unit 22, and a server controller 23.

The server communication unit 21 includes a communication moduleconnected to the network 30. The communication module is compatiblewith, for example, a wired local area network (LAN) standard, but is notlimited thereto, and may be compatible with any communication standard.In the embodiment, the server 20 is connected to the network 30 throughthe server communication unit 21.

The server storage unit 22 includes one or more memories. Each memoryincluded in the server storage unit 22 may function as, for example, amain storage device, an auxiliary storage device, or a cache memory. Theserver storage unit 22 stores predetermined information used for theoperation of the server 20. For example, the server storage unit 22 maystore a system program, an application program, a database, road mapinformation and the like. The information stored in the server storageunit 22 may be updatable with, for example, information to be acquiredfrom the network 30 through the server communication unit 21.

The server controller 23 includes one or more processors. The servercontroller 23 controls the operation of the entire server 20.

For example, the server controller 23 collects information from one ormore vehicles 10. Specifically, the server controller 23 receives theroad ID, the imaging time and the degree of difficulty in traveling foreach of one or more roads from one or more vehicles 10 through theserver communication unit 21. The server controller 23 stores thereceived information in the server storage unit 22. Specifically, forexample, as shown in FIG. 5, the degree of difficulty in traveling andthe like for each of one or more roads are stored in the database of theserver storage unit 22. As described above, when the vehicle 10transmits the image to the server 20, the server controller 23 may alsostore the image in the database of the server storage unit 22.

In addition, the server controller 23 provides information to the client31 by using the degree of difficulty in traveling and the like for eachof the one or more roads, stored in the server storage unit 22. Theprovision of information may be performed, for example, in response tothe request from the client 31 (for example, pull distribution), or maybe automatically performed by the server controller 23 (for example,push distribution). The provision of information may be performed by aweb application stored in the server storage unit 22. The provision ofinformation performed by using the degree of difficulty in traveling andthe like for each of the one or more roads may include providing atleast part of the information such as the degree of difficulty intraveling stored in the server storage unit 22 as it is, or may includeproviding any information to be newly generated by using the degree ofdifficulty in traveling and the like.

For example, FIG. 6 is a diagram showing an example of a screendisplayed to the client 31 based on the information provided from theserver 20. In the screen shown in FIG. 6, the degree of difficulty intraveling on the road, which is indicated by an arrow on a map, isdisplayed. The user of the client 31 can grasp the degree of difficultyin traveling on the road at a glance by visually recognizing the screen.As described above, when the vehicle 10 transmits the image to theserver 20, the screen displayed on the client 31 may display the imageand the degree of difficulty in traveling on the road indicated by thearrow on the map. In addition, for example, a configuration is alsopossible in which the arrow is displayed on the map and then the imageand the degree of difficulty in traveling on the road indicated by thearrow are displayed according to a user operation for selecting thearrow.

Operation Flow of Vehicle

An operation flow of the vehicle 10 will be described with reference toFIG. 7.

Step S100: The controller 15 acquires weather information indicating asnow cover area or a snowfall area.

Step S101: The controller 15 determines whether or not the host vehicleis located in the snow cover area or the snowfall area indicated by theacquired weather information. When determination is made that the hostvehicle is not located in the snow cover area or the snowfall area (stepS101—No), the process ends. On the other hand, when determination ismade that the host vehicle is located in the snow cover area or thesnowfall area (step S101—Yes), the process proceeds to step S102.

Step S102: The controller 15 acquires an image obtained by imaging theroad on which the host vehicle is located, an imaging time of the image,and a road ID of the road.

Step S103: The controller 15 determines the degree of difficulty intraveling on the road due to snow cover from the image in step S102.

Step S104: The controller 15 transmits the above-mentioned road ID, theimaging time, and the degree of difficulty in traveling to the server 20through the communication unit 11.

Operation Flow of Server

An operation flow of the server 20 will be described with reference toFIG. 8.

Step S200: The server controller 23 collects information from one ormore vehicles. Specifically, the server controller 23 receives the roadID, the imaging time and the degree of difficulty in traveling for eachof one or more roads from one or more vehicles 10 through the servercommunication unit 21.

Step S201: The server controller 23 stores information received in StepS200 (the road ID, the imaging time, and the degree of difficulty intraveling for each of the one or more roads) in the server storage unit22.

Step S202: The server controller 23 provides information to the client31 by using the degree of difficulty in traveling and the like for eachof the one or more roads, stored in the server storage unit 22.

As described above, in the information processing system 1 according tothe embodiment, the vehicle 10 acquires the image obtained by imagingthe road on which the host vehicle is located, and determines the degreeof difficulty in traveling on the road due to snow cover from the image.The server 20 stores the degree of difficulty in traveling for each ofone or more roads, and provides information to the client 31 by usingthe degree of difficulty in traveling for each of one or more roads.With such a configuration, the degree of difficulty in traveling due tosnow cover is determined by using the image that has been actuallyimaged by the vehicle 10. Therefore, it is possible to obtain a moreaccurate degree of difficulty in traveling that is in line with theactual situation in the field than the degree of difficulty in travelingestimated from weather information, for example. Therefore, thetechnique for acquiring information on snow cover on a road is improved.

The disclosure has been described based on the drawings and theexamples, but it is to be noted that those skilled in the art easilyperform various modifications and changes based on the disclosure.Therefore, it is to be noted that the modifications and changes areincluded in the scope of the disclosure. For example, the functions andthe like included in each unit, each step, or the like can be disposedagain so as not to be logically contradictory, and a plurality of means,steps, or the like can be combined into one, or divided.

For example, in the embodiment described above, some processingoperations executed in the vehicle 10 may be executed in the server 20,and some processing operations executed in the server 20 may be executedin the vehicle 10. For example, it is possible to adopt a configurationin which the determination processing for determining, from the imageobtained by imaging the road, the degree of difficulty in traveling dueto snow cover is made by the server 20, instead of the vehicle 10. Inthis case, the vehicle 10 transmits the road ID of the road, the imagingtime, and the image to the server 20. The server 20 performs theabove-mentioned determination by using the image received from thevehicle 10.

In addition, in the embodiment described above, the server 20 has beendescribed to have a configuration of providing information to the client31 by using the degree of difficulty in traveling for each of the one ormore roads. However, information on the degree of difficulty intraveling is not limited to being used for providing information to theclient 31, and may be used for any other purposes. For example, when thevehicle 10 can perform the autonomous driving, the controller 15 of thevehicle 10 may perform the autonomous driving by using the degree ofdifficulty in traveling determined by the host vehicle or received fromthe server 20. Specifically, when the degree of difficulty in travelingis equal to or greater than a predetermined reference value, aconfiguration is conceivable in which the detour of road is performed, atraveling route to a destination is determined by preferentially using aroad with a low degree of difficulty in traveling, or the upper limitvehicle speed is made slower as the degree of difficulty in travelingincreases.

A general-purpose information processing devices such as a smartphone, acomputer, or the like can be configured to function as a configurationunit provided in the vehicle 10 according to the embodiment describedabove or the server 20. Specifically, a program, in which processingcontents for realizing each function of the vehicle 10, the server 20 orthe like according to the embodiment are described, is stored in amemory of the information processing device such that a processor of theinformation processing device reads and executes the program. Therefore,the disclosure according to the embodiment can also be realized as theprogram that can be executed by the processor.

What is claimed is:
 1. An information processing system comprising: oneor more vehicles; and a server communicable with the one or morevehicles, wherein: a vehicle of the one or more vehicles is configuredto: acquire an image obtained by imaging a road on which the vehicle islocated; receive positioning signals from a satellite positioningsystem; determine a position of the vehicle when the image was obtainedbased upon the positioning signals; the vehicle or the server isconfigured to execute determination processing for determining a degreeof difficulty in traveling on the road due to snow cover from the imageincluding determining that the degree of difficulty in traveling on theroad is lower when at least two of four detection targets is detectedfrom the image than when no detection target is detected, the fourdetection targets including another vehicle traveling on the road, aroad marking on the road, a tire trace on the road, and snow piled upbeside the road; and the server is configured to: store the degree ofdifficulty in traveling for each of one or more roads, and provideinformation to a client by using the stored degree of difficulty intraveling for each of the one or more roads.
 2. The informationprocessing system according to claim 1, wherein the vehicle isconfigured to: acquire weather information indicating a snow cover areaor a snowfall area; and acquire the image when the vehicle is located inthe snow cover area or the snowfall area indicated by the weatherinformation.
 3. The information processing system according to claim 1,wherein the server provides road map information including degrees ofdifficulty in traveling on the one or more roads to the client in theproviding of information.
 4. The information processing system accordingto claim 1, wherein the vehicle is configured to control driving of thevehicle based upon the degree of difficulty in traveling.
 5. Aninformation processing system comprising: one or more vehicles; and aserver communicable with the one or more vehicles, wherein: a vehicle ofthe one or more vehicles is configured to: acquire an image obtained byimaging a road on which the vehicle is located; receive positioningsignals from a satellite positioning system; determine a position of thevehicle when the image was obtained based upon the positioning signals;the vehicle or the server is configured to execute determinationprocessing for determining a degree of difficulty in traveling on theroad due to snow cover from the image including determining that thedegree of difficulty in traveling on the road is lower when at leastthree of four detection targets is detected from the image than when nodetection target is detected, the four detection targets includinganother vehicle traveling on the road, a road marking on the road, atire trace on the road, and snow piled up beside the road; and theserver is configured to: store the degree of difficulty in traveling foreach of one or more roads, and provide information to a client by usingthe stored degree of difficulty in traveling for each of the one or moreroads.
 6. The information processing system according to claim 5,wherein the vehicle is configured to: acquire weather informationindicating a snow cover area or a snowfall area; and acquire the imagewhen the vehicle is located in the snow cover area or the snowfall areaindicated by the weather information.
 7. The information processingsystem according to claim 5, wherein the server provides road mapinformation including degrees of difficulty in traveling on the one ormore roads to the client in the providing of information.
 8. Theinformation processing system according to claim 5, wherein the vehicleis configured to control driving of the vehicle based upon the degree ofdifficulty in traveling.
 9. An information processing system comprising:one or more vehicles; and a server communicable with the one or morevehicles, wherein: a vehicle of the one or more vehicles is configuredto: acquire an image obtained by imaging a road on which the vehicle islocated; receive positioning signals from a satellite positioningsystem; determine a position of the vehicle when the image was obtainedbased upon the positioning signals; the vehicle or the server isconfigured to execute determination processing for determining a degreeof difficulty in traveling on the road due to snow cover from the imageincluding determining that the degree of difficulty in traveling on theroad is lower when all of four detection targets is detected from theimage than when no detection target is detected, the four detectiontargets including another vehicle traveling on the road, a road markingon the road, a tire trace on the road, and snow piled up beside theroad; and the server is configured to: store the degree of difficulty intraveling for each of one or more roads, and provide information to aclient by using the stored degree of difficulty in traveling for each ofthe one or more roads.
 10. The information processing system accordingto claim 9, wherein the vehicle is configured to: acquire weatherinformation indicating a snow cover area or a snowfall area; and acquirethe image when the vehicle is located in the snow cover area or thesnowfall area indicated by the weather information.
 11. The informationprocessing system according to claim 9, wherein the server provides roadmap information including degrees of difficulty in traveling on the oneor more roads to the client in the providing of information.
 12. Theinformation processing system according to claim 9, wherein the vehicleis configured to control driving of the vehicle based upon the degree ofdifficulty in traveling.